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ABSTRACT 



A project was conducted to revise the list of assessment 
instruments that Pennsylvania Department of Education, Bureau of Adult Basic 
and Literacy Education (ABLE) -funded agencies are permitted to use to report 
learner gains. Through a mail survey, the project researched 19 standardized 
assessment instruments used by agencies for reporting learner gains and 
identified areas in which appropriate instruments were needed. Instruments 
that were in use and others that were identified as having potential use were 
examined to determine whether they met acceptability requirements. 
Recommendations for revising the list of acceptable instruments were 
submitted to the bureau. The project found that five standardized 
instruments- -the TABE (Tests of Adult Basic Education) Survey Edition, the 
TABE Complete Battery, the Official GED (General Educational Development) 
Practice Tests, the GED Tests, and the BEST (Basic English Skills Test) --are 
the most popular instruments among responding agencies, and that respondents 
are generally satisfied with those instruments. A need was identified for 
standardized instruments that would supply better information on 
beginning -level readers, career-specific learners, and upper-level ESL 
(English as a Second Language) learners. (Appendixes to the report include 
the survey and lists of assessment instruments and publishers.) (Author/KC) 
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ABSTRACT 



Project No. 98-99-9012 



Grant Recipient: 

Carol Molek 

TIU Adult Education and Job Training Center 
MCIDC Plaza, Building 58, 6395 SR103 North 
Lewistown, PA 17044 
(717) 248-4942 

Program Name: ABLE Assessment Practices 
Grant Allocation: $27,901 
Project Period: July 1 , 1998 - June 30, 1999 
Project Director: Carol Molek 

Project Purpose: This project provided data to the Pennsylvania Department of 
Education, Bureau of Adult Basic and Literacy Education, as it revised its list of 
assessment instruments allowable for use by ABLE-funded agencies in reporting learner 
gains to the Bureau. The project researched the standardized instruments used by agencies 
for reporting learner gains and identified areas where appropriate instruments were needed. 
Instruments that were in use and others that were identified as good candidates for use by 
agencies for the purpose of reporting learner gains were examined to determine whether 
they met requirements for being acceptable reporting instruments. Recommendations for 
revising the list of acceptable instruments for reporting were submitted to the Bureau. 

Project Outcomes: A mail survey of ABLE-funded agencies found that five 
standardized instruments — The TABE Survey Edition, the TABE Complete Battery 
Edition, the Official GED Practice Tests, the GED Tests, and the BEST — are the most 
popular instruments among responding agencies for reporting learner gains. Respondents 
are generally satisfied with the instruments they are using for reporting learner gains, 
especially when they are supplemented with other standardized or informal instruments. A 
need was identified for standardized instruments that would supply better information on 
beginning-level readers, career-specific learners, and upper-level ESL learners. 

Impact: The project reviewed 19 assessment instruments to determine whether they met 
requirements for being acceptable reporting instruments. Special attention was given to 
locating instruments to meet the needs of the three learner groups identified above. 
Recommendations for revising the list of acceptable instruments based on the research and 
instrument reviews were submitted to the Bureau for review and consideration. 

Products or Training Developed: A final report summarizes project activities, 
research findings, and recommendations. 

Products Available From: the Ad vancE State Literacy Resource Center. 

Project Continuation and/or Future Implications: The project provides standards 
for review of instruments that may be useful in the future as instruments are revised or new 
ones become available. 

Conclusions/Recommendations: At this point in time, the list of instruments 
appropriate for reporting learner gains is short. In the future, other possibilities in new or 
revised instruments should continue to be explored. 



Additional Comments: 



Introduction 



Purpose and Objectives of the Project 

“ABLE Assessment Practices” provided data and information to the 
Pennsylvania Department of Education, Bureau of Adult Basic and Literacy Education, 
as it began the process of revising its list of assessment instruments allowable for use 
by ABLE-funded agencies in reporting learner gains to the Bureau. The project 
objectives were: 

1. To research, through a mail survey to ABLE-funded agencies, the standardized 
instruments currently being used in their service-providing programs, and to report 
instruments, publishers, and the number and type of agency (e.g., volunteer, school 
district, community-based organization, etc.) using each instrument; 

2. To research, through a follow-up telephone survey with select agencies responding 
to the mail survey, why agencies are using particular instruments for pre- and post- 
testing, identify needs and constraints guiding the selection of assessment instruments, 
identify gaps or areas where appropriate instruments are critically needed, and to report 
these findings; 

3. To gather information on standardized assessments that meet the Bureau’s 
requirements and that may be appropriate for populations served by ABLE programs, 
including literacy, ABE, GED, ESL, and work-focus; 

4. To examine potential instruments, and those that are being used in agencies, to 
determine their appropriateness of use related to the Bureau’s requirements; 

5. To develop a revised listing of acceptable standardized instruments, to be used by 

agencies for reporting educational gains, for approval and dissemination by the Bureau. 
Rationale and Background of the Project ( 

The ABLE Bureau requires that agencies with Act 143 and/or Section 322 
funding administer standardized pre- and post-tests to cooperating students and report 
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the scores to the Bureau. This data, as an indicator of learner gains, is part of the 
information used to evaluate individual agency and overall state efforts to increase skills 
of the adults served by these programs. The Bureau requires that the same instrument 
be used to pre- and post-test a particular student. Furthermore, the permissible 
instruments are those that are standardized on an appropriate adult population and 
include at least two different but equivalent forms (“Instructions: Adult Basic & 
Literacy Education Data Forms,” July, 1998). Currently, ABLE program performance 
standards state that each provider must submit matched pre- and post-test scores for at 
least 35% of enrolled students. The requirements that the Bureau sets for acceptable 
reporting instruments are necessary because of the way that the data is used: to make 
comparisons of gain across students and programs. Other standardized and non- 
standardized instruments play very important roles in the work that programs do. The 
Bureau encourages the use of a variety of assessment tools to meet various purposes 
within programs. 

There has been some confusion in the field over which instruments are 
appropriate to use for the purposes of reporting learner gains. In the recent past the 
Bureau accepted alternative methods of reporting learner gains, including published 
informal assessments and teacher- or program-developed assessments. However, 
more recently, evolution of the field has increased the need for accountability and the 
Bureau has gradually increased the stringency of guidelines for demonstrating learner 
gains. During this transitional time, the Bureau’s list of acceptable assessment 
instruments (see the July, 1998 “Instructions: Adult Basic & Literacy Education Data 
Forms”) included some that are not standardized (e.g., ESLOA/LVA ESL Oral 
Assessment), others that, although standardized, are not normed on adults (e.g., the 
Secondary Level English Proficiency Test), and those that, as locator tests, are not 
appropriate for reporting pre- and post-test scores to document learner gains (e.g. the 
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TABE Locator). Many agencies are looking for instruments that meet the Bureau’s 
requirements and mesh with their program’s instructional philosophies, learner 
characteristics, and learner goals. The Bureau is seeking to refine the list of 
instruments that are acceptable for reporting learner gains to the state while including 
instruments that meet the needs of agencies and the learners they serve. The Bureau 
commissioned this project in order to collect data to guide decision-making regarding 
the revamping of the list 

The project extended previous research on assessment done in Pennsylvania 
(Adult Education Linkage Services, 1991; Zellers, 1986) in three ways. First, it 
focused on standardized instruments and the extent to which they meet the Bureau’s 
current requirements of being standardized on an adult population and including at least 
two equivalent forms for reporting educational gains. Second, it gathered updated 
information on standardized assessments to determine their appropriateness. For 
example, a revised version of the Tests of Adult Basic Education (TABE), a widely- 
used assessment in ABLE programs, has recently been introduced with some 
significant changes from the previous version; this would not be represented in either 
previous project Third, unlike the previous projects, the current project also 
considered numerous populations served by ABLE-funded service providers, including 
literacy, Adult Basic Education (ABE), General Educational Development (GED), 
English as a Second Language (ESL), and work-focus. 

Project Time Frame 

During Fall, 1998, a brief mail survey on use of assessment instruments was 
developed and distributed to 262 directors on the Bureau’s mailing list of agencies 
receiving instructional funds. Survey results were analyzed and summarized during the 
early months of 1999. Information on lesser-used instruments that may be appropriate 
for tracking learner gains for various populations served by ABLE (literacy, ABE, 
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GED, ESL, and work-focus) was located and collected throughout the project. Follow- 
up telephone interviews were conducted with a subset of agencies that had responded to 
the mail survey at the end of the project year, so that project staff could capture the 
experiences that the respondents had with the instruments during the project year. In 
late Spring, 1999, instruments were targeted for in-depth review based on results of the 
mail survey, telephone interviews, and search for instruments. In late Spring and early 
Summer 1999, reviews of the instruments were completed, recommendations to the 
Bureau were summarized, and the Final Report was compiled. 

Project Staff and Key Personnel 

The Project Director was Carol Molek. Ms. Molek has over 15 years 
experience directing adult programs for the Tuscarora Intermediate Unit and developing 
and implementing special projects. She has successfully directed numerous Section 
353 projects for the Bureau, several of those focusing on research on and development 
of assessment tools and practices. She co-authored, along with Lori Forlizzi and 
Barbara Van Horn, the “Assessment for Adult Learners” training module being 
delivered throughout the state as part of the ABLE-funded Training Module Project. 

The module is an overview of critical concepts and information regarding assessment in 
the adult education environment. She directed all project activities, oversaw fiscal 
management of the project, and reviewed the mail survey, draft Final Report, and 
recommendations to the Bureau. The Project Coordinator was Lori Forlizzi. She had 
prior experience conducting survey and telephone interview research and research- 
focused 353 projects in Pennsylvania, including a statewide survey of ABLE service 
provision during the 1993 - 1994 project year. She co-authored the “Assessment for 
Adult Learners” training module and has served as the statewide lead assessment trainer 
for the module as well as an assessment trainer for the South Central Professional 
Development Center. She has also served as a statewide resource person on 
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assessment issues. She developed the mail survey, analyzed and summarized the 
results, conducted the follow-up telephone interviews, selected instruments for review, 
reviewed the instruments, drafted recommendations to the Bureau, and wrote the Final 
Report Ms. Sharon Fisher, a staff assistant at the Adult Education and Job Training 
Center, oversaw ordering of assessment materials for review. 

Audience for this Report 

The audience for this report primarily includes staff of the ABLE Bureau and 
staff of ABLE-funded programs. 

Project Dissemination 

Copies of this report will be filed permanendy at the following locations: 

1) Pennsylvania Department of Education 

Bureau of Adult Basic and Literacy Education 
333 Market Street, 12th Floor 
Harrisburg, PA 17126-0333 

2) AdvancE State Literacy Resource Center 

333 Market Street, 1 1th Floor 
Harrisburg, PA 17126-0333 

Statement of the Problem 

“ABLE Assessment Practices” provided data and information to the 
Pennsylvania Department of Education, Bureau of Adult Basic and Literacy Education, 
as it began the process of revising its list of instruments allowable for use by ABLE- 
funded agencies in reporting learner gains to the Bureau. 

Project Goals and Objectives 
Project objectives were: 

1. To research, through a mail survey to ABLE-funded agencies, the standardized 
instruments currently being used in their service-providing programs, and to report 
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instruments, publishers, and the number and type of agency (e.g., volunteer, school 
district, community-based organization, etc.) using each instrument; 

2. To research, through a follow-up telephone survey with select agencies responding 
to the mail survey, why agencies are using particular instruments for pre- and post- 
testing, identify needs and constraints guiding the selection of assessment instruments, 
identify gaps or areas where appropriate instruments are critically needed, and to report 
these findings; 

3. To gather information on standardized assessments that meet the Bureau’s 
requirements and that may be appropriate for populations served by ABLE programs, 
including literacy, ABE, GED, ESL, and work-focus; 

4. To examine potential instruments, and those that are being used in agencies, to 
determine their appropriateness of use related to the Bureau’s requirements; 

5. To develop a revised listing of acceptable standardized instruments, to be used by 
agencies for reporting educational gains, for approval and dissemination by the Bureau. 

Procedures Employed 

“ABLE Assessment Practices” was conducted in two parts. During Part I, a 
brief mail survey was designed that gathered information on published assessment 
instruments were using to report learner gains and for other purposes. The survey was 
mailed to directors of 262 ABLE agencies receiving instructional funds. A follow-up, 
reminder mailing in November was completed in order to increase the survey response 
rate. Also, during Part I, information on lesser-used instruments that may be 
appropriate for tracking learner gains for various populations served by ABLE (literacy, 
ABE, GED, ESL, and work-focus) was gathered through a variety of strategies 
described in a later section of this report. During Part II, survey results were analyzed 
and summarized. Follow-up telephone interviews were conducted with 13% of 
agencies that had responded to the mail survey at the end of the project year, so that 
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project staff could capture the experiences that the respondents had with the instruments 
during the project year. The agencies interviewed were selected to represent the types 
and geographical locations of agencies responding to the mail survey. Also during Part 
II, instruments were targeted for in-depth review based on results of the mail survey, 
telephone interviews, and search for instruments. Reviews of the instruments were 
completed, recommendations to the Bureau were summarized, and the Final Report 
was compiled. 

Objectives Achieved 

Objective 1: To research, through a mail survey to ABLE-funded 
agencies, the standardized instruments currently being used in their 
service-providing programs, and to report instruments, publishers, and 
the number and type of agency (e.g., volunteer, school district, 
community-based organization, etc.) using each instrument. 

In October, 1998, a survey to gather information on published assessment 
instruments then being used by ABLE agencies for reporting learner gains to the state 
or for other purposes was mailed to 262 directors on the Bureau’s mailing list of 
agencies receiving instructional funds (a copy of the survey is included in Appendix A). 
By the end of November, only 68% of the agencies had responded. To increase the 
response rate, a second mailing of the survey to non-responding agencies was 
conducted in November 1998. Two-hundred and eighteen agencies, or a total of 83% 
of the agencies contacted, eventually responded to the survey (two agencies indicated 
they were no longer conducting programs). The following analyses thus include the 
data from 216 responding ABLE programs. 

Types of agencies responding to the survey. Agencies were asked to 
identify themselves as one of the following types: literacy council, school district, 
intermediate unit, library, community-based organization, community college, or 
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“other.” The categories that respondents listed under “other” were numerous, including 
institution, university, state correctional institution, vocational-technical school, 
college, library literacy program, county prison, post-secondary trade-technical school, 
community corrections, alternative high school, non-profit organization, union-based 
organization, correctional facility, long-term health care facility, social service agency, 
and public housing. The table in Appendix B shows the number and percentage of 
respondents identifying themselves under each category. Not surprisingly, the greatest 
proportion of respondents (30%) identified themselves as community-based 
organizations, followed by school districts (19%), literacy councils (10%), state 
correctional institutions (10%), and intermediate units (9%). 

Published assessment instruments used for reporting learner gains 
to the state. The survey asked respondents to indicate which of nine published 
instruments or systems they use for reporting learner gains to the state, including the 
Tests of Adult Basic Education (TABE) Survey Edition, the TABE Complete Battery 
Edition, the Basic English Skills Test (BEST), the Adult Basic Learning Examination 
(ABLE), Comprehensive Adult Student Assessment System (CASAS) assessments, 
the Official GED Practice Tests, the GED Tests, Reading Evaluation Adult Diagnosis 
(READ), and the English as a Second Language Oral Assessment (ESLOA). Agencies 
were also given the opportunity to write in the names of other instruments being used 
for this purpose and were asked to include the name of the publisher of each instrument 
that they wrote in. 

Project staff decided to collect data on all instruments that agencies were using 
to report learner gains, and then to examine each in order to make a determination as to 
whether it was standardized. This procedure spared respondents from making their 
own determination as to whether the instruments they were using were standardized, 
and allowed the project to capture the range of instruments actually being used for 
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reporting learner gains whether they were standardized or not. Therefore, both 
standardized and informal instruments thought to be commonly used by ABLE-funded 
agencies were included on the survey. 

Two-hundred and twelve, or 98% of respondents, indicated that they were 
using at least one published instrument to report learner gains to the state. Appendix C 
shows the instruments used to report learner gains, whether or not the instrument(s) are 
standardized, the number, type, and percentage of educational service agencies using 
each instrument, and the publishers of those instruments. (Note: the information in the 
Appendix regarding publisher and standardization status reflects the most recent edition 
of the instrument or instruments; data provided by respondents was not always clear 
about which edition of an instrument was being used, or in the case of CASAS and the 
Brigance assessments, which particular instrument was being used). Since numerous 
agency types were indicated by respondents, only the top nine agency types are 
included in this table; the table accounts for 92% of the respondents. Seventeen 
instruments or systems that at least three agencies indicated they are using are included 
in the table. A list of other instruments named by at least one but fewer than three 
agencies is included in Appendix D. Often, complete names were not given for these 
instruments, nor were names of publishers given, so it is difficult to make sense of 
some of the responses included in the list. Where possible, names of publishers are 
included in the list 

By far the most popular instrument for reporting learner gains across agency 
types included in the table is the Official GED Practice Tests. This was the top reported 
instrument by all agency types except state correctional institutions and universities. 

The most popular instruments for reporting learner gains appear to be the Official GED 
Practice Tests, the GED Tests, the TABE Survey and Complete Battery, and the BEST. 
CASAS assessments and the ABLE are used to a lesser extent for this purpose. The 
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Wide Range Achievement Test (WRAT) is used by a small number of most agency 
types. 

The most popular instruments for reporting learner gains indicated by 
responding school districts were the Official GED Practice Tests (78%), the GED Tests 
(54%), and the TABE Survey and Complete Battery Editions (both at 29%). These 
four instruments were also those most likely to be used by state correctional 
institutions, intermediate units, and vocational-technical schools as well. The 
instruments most frequendy indicated by responding state correctional institutions 
included the TABE Complete Battery Edition (86%) followed by the TABE Survey 
Edition, the Official GED Practice Tests, and the GED Tests, each at 68%. Those 
indicated by responding intermediate units included the GED Practice Tests, the GED 
Tests, and TABE Survey Edition (all at 58%), and the TABE Complete Battery Edition 
(42%). Those indicated by responding vocational-technical schools included the 
Official GED Practice Tests (100%), followed by the TABE Survey and Complete 
Battery Editions and the GED Tests (all at 33%). 

Community-based organizations, school districts, and literacy councils tended 
to report using the widest variety of instruments for reporting learner gains. The BEST 
was a popular choice for agencies other than school districts, state correctional 
institutions, intermediate units, and vocational-technical schools. The most popular 
instruments for reporting learner gains indicated by responding community-based 
organizations were the Official GED Practice Tests (44%), The GED Tests and the 
BEST (both at 28%) and the TABE Complete Battery Edition (23%). Those indicated 
by responding literacy councils include the BEST and the Official GED Practice Tests 
(both 67%), followed by the TABE Survey Edition (48%) and the GED Tests (38%). 
For libraries the TABE Survey Edition and the GED Practice Tests (both at 44%) were 
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followed in popularity by the BEST, the GED Tests, and the Slosson Oral Reading 
Test (SORT), all at 22%. 

The most popular instruments for reporting learner gains indicated by 
responding community colleges included The Official GED Practice Tests and the GED 
Tests (both at 73%) followed by the TABE Complete Battery Edition (45%) and the 
BEST (36%). Those indicated by responding universities included the TABE 
Complete Battery Edition (80%), followed by the BEST, The Official GED Practice 
Tests, and the GED Tests, all at 40%. 

As previously noted, the instruments that respondents indicated they were using 
to report learner gains were reviewed after the survey results were in to make a 
determination as to whether or not they were standardized. An instrument was 
considered “standardized” if it requires administration under a standard set of 
conditions (including standard stimulus materials, items, and procedures), the use of 
consistent procedures for scoring and interpreting the results, if it has been normed on a 
substantially large group of individuals, if it provides standard (scaled) scores, and if it 
has demonstrated reliability and validity. Seven of the instruments/systems included in 
the table were not reviewed for this project: the TABE Complete Battery, the TABE 
Survey, the ABLE, the BEST, the CASAS assessments, the Official GED Practice 
Tests, and the GED Tests. This is because it is widely known that these instruments 
are psychometrically sound, standardized instruments widely used with adult learners. 
The CASAS is actually a system of instruments including informal and alternative 
assessments. While the survey was not powerful enough to determine which of the 
CASAS assessments were being used by respondents for reporting learner gains, the 
“introductory” systems promoted by CASAS and introduced through basic training 
required to purchase and use CASAS assessments (the Life Skills System 
Survey/Achievement Pre and Post Tests and the Employability Competency System 
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Survey/Achievement Pre and Post Tests) — those most likely to be used by agencies 
who are in the early adoption stages of CASAS — are standardized. 

The other ten instruments/systems included in Appendix C were reviewed as 
part of this project to confirm whether or not they were standardized. These were the 
READ, the ESLOA, the WRAT-III, the Wonderlic Basic Skills Test (WBST), the 
Secondary Level English Proficiency Test (SLEP), the Woodcock Reading Mastery 
Tests-Revised (WRMT-R), The Adult Measure of Essential Skills (AMES), the 
Slosson Oral Reading Test-Revised (SORT-R), and the Brigance Inventories that are 
appropriate for use with adults: the Brigance Diagnostic Inventory of Essential Skills; 
the Brigance Diagnostic Life Skills Inventory; and the Brigance Diagnostic 
Employability Skills Inventory. The “Contemporary GED” tests category that came out 
of the survey included curriculum-based assessments as well as Contemporary’s 
“Diagnostic Pre-tests for GED Instruction” and “Evaluative Post-tests for GED 
Readiness.” It is widely known that the curriculum-based assessments included in 
these instructional series are not standardized; however, the diagnostic pre-tests and 
evaluative post-tests were examined for the project. As previously noted, the status of 
each instrument in terms of standardization is indicated in the table in Appendix C. 

The top five instruments used for reporting learner gains as indicated by 
respondents to this survey are standardized instruments (the Official GED Practice 
Tests, the GED Tests, the TABE Survey and Complete Battery, and the BEST). Many 
of the others used for this purpose are standardized as well (e.g., CASAS, ABLE, the 
WRAT-III, the WBST, the SLEP, the WRMT-R, the AMES, and the SORT-R. 
However, it should be noted that not all of the instruments that respondents indicated 
on this survey that they are using to report learner gains are standardized: these include 
the READ, the ESLOA, the three Brigance inventories, and the Contemporary 
diagnostic pre-tests and evaluative post-tests. The survey results may overestimate the 
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number of non-standardized instruments being used to report learner gains. Follow-up 
interviews with 13% of the responding agencies (see Objective 2) indicated that at least 
some responding agencies had indicated use of non-standardized instruments at the 
beginning of the project year but during the year had discontinued use of these and 
adopted standardized instruments to be in line with the Bureau’s requirements. 

This objective explored whether the instruments being used by agencies to 
report learner gains are standardized instruments. The question of whether the 
standardized instruments being used to report learner gains are suitable for use with the 
adult populations of ABLE-funded agencies in relation to the Bureau’s requirements 
was explored through Objective 4. 

Published assessment instruments used for other purposes. The 
survey also asked respondents to indicate which of nine published instruments they use 
for purposes other than reporting learner gains to the state, including the TABE Survey, 
the TABE Complete Battery, the BEST, the ABLE, CASAS assessments, the Official 
GED Practice Tests, the GED Tests, the READ, and the ESLOA. They were again 
given the opportunity to write in the names of other instruments and were asked to 
include the name of the publisher of each instrument that they wrote in. 

One hundred and seventy-one agencies, or 79% of respondents, indicated that 
they were using at least one published instrument for purposes other than reporting 
learner gains to the state. Appendix E shows the instruments being used for other 
purposes, and the number, type, and percentage of educational service agencies using 
each instrument. (The publishers of those instruments are shown in the Key in 
Appendix C). Again, only the top nine agency types are included in the table, 
representing 92% of the responding agencies. Instruments that at least three agencies 
indicated they are using are included in the table. A list of other instruments named by 
at least one but fewer than three agencies is included in Appendix F. Since complete 
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names of instruments and publishers were often not provided by respondents, the list is 
as complete and accurate as possible. 

Again, the most popular instrument used for purposes other than reporting 
learner gains is the Official GED Practice Tests. This was the top reported instrument 
by all agency types. The most popular instruments used for purposes other than 
reporting learner gains appear to be, once again, the Official GED Practice Tests, the 
GED Tests, and the TABE Survey and Complete Battery Editions. It appears that the 
BEST, ABLE, and CASAS assessments are not widely used for purposes other than 
reporting learner gains to the state. The SORT is reportedly used by almost a quarter of 
responding libraries and universities. The WRAT is again reportedly used by a small 
number of most agency types. Nineteen percent of responding literacy councils also 
report using the ESLOA for purposes other than reporting learner gains. 

The most popular instruments for “other purposes” reported by school districts 
were the Official GED Practice Tests (49%) followed by the GED Tests (27%) and the 
TABE Survey Edition (22%). The instruments most frequendy indicated by 
responding state correctional institutions included The Official GED Practice Tests 
(95%), The GED Tests (91%), and the TABE Survey Edition (86%). Those indicated 
by responding intermediate units include the GED Practice Tests (63%), the GED Tests 
(53%), and the TABE Survey Edition (42%). Those indicated by responding 
vocational-technical schools included The Official GED Practice Tests (50%) and the 
TABE Complete Battery Edition (50%). School districts, intermediate units, 
vocational-technical schools and state correctional institutions rely on the same four 
instruments — the TABE Survey Edition, the TABE Complete Battery Edition, the 
Official GED Practice Tests, and the GED Tests — for reporting learner gains and for 
other purposes. 
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Community-based organizations indicated that The Official GED Practice Tests 
(55%) and the GED Tests (31%) were used for purposes other than reporting learner 
gains. Literacy councils indicated using The Official GED Practice Tests (38%), the 
TABE Survey Edition (33%), and the GED Tests (29%). For libraries, The GED 
Practice Tests (44%) and the GED Tests (33%) were followed by the TABE Survey 
Edition, the BEST, and SORT all at 22%. Community colleges indicated using the 
GED Practice Tests and the TABE Complete Battery Edition (both at 45%), followed 
by the GED Tests at 36%. Universities indicated use of the Official GED Practice Tests 
(60%), followed by the TABE Complete Battery Edition and the SORT (both at 20%). 
Community-based organizations and school districts tended to report using the widest 
variety of instruments for purposes other than reporting learner gains. For agencies 
other than school districts, intermediate units, vocational-technical schools and state 
correctional institutions, the SORT and the BEST appeared as lesser-used choices. 

Summary. The most remarkable thing about these results is the extent to 
which the pattern of assessment instrument usage is similar across types of agencies 
and purposes (either reporting learner gains to the state or other purposes). Four 
instruments ~ the TABE Survey Edition, the TABE Complete Battery Edition, the GED 
Tests, and by far the most often, the Official GED Practice Tests, appear to be used by 
most agencies for most purposes. The most popular instruments for reporting learner 
gains appear to be the Official GED Practice Tests, the GED Tests, the TABE Survey 
and Complete Battery, and the BEST. A “second tier” in popularity for reporting 
learner gains would include the ABLE, CASAS, and the WRAT. The TABE Survey 
and Complete Battery, Official GED Practice Tests, and the BEST, along with the 
ABLE and CASAS, are included in the basic 12 hour “Assessment for Adult Learners” 
training module that is required of all ABLE-funded agencies and are accepted by the 
Bureau for reporting learning gains. It appears that ABLE-funded agencies have 
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widely adopted several of the appropriate and highly publicized instruments. The 
results should be interpreted cautiously, keeping in mind that the data resulted from 
agency self-reports and did not include all ABLE-funded agencies. 

Objective 2: To research, through a follow-up telephone survey with 
select agencies responding to the mail survey, why agencies are using 
particular instruments for pre- and post-testing, identify needs and 
constraints guiding the selection of assessment instruments, identify 
gaps or areas where appropriate instruments are critically needed, and to 
report these findings. 

The proposal indicated that the Project Coordinator would conduct telephone 
interviews with 5 - 10% of survey respondents (216 agencies), or between 1 1 and 22 
agencies. Telephone surveys were conducted at the end of the project year in order to 
capture respondents’ year of experience with the instruments. Telephone contacts were 
made with 58 agencies and a total of 29 interviews were completed, representing 13% 
of responding agencies (a large number of calls were made to ensure the targeted 
number of responses, as some individuals contacted were not available or did not 
respond to a request for a call back). The Project Coordinator set out to contact a 
sample of agencies proportionally representing the nine agency types appearing in the 
tables in Appendices C and E. Thus, since 30% of the group of responding agencies 
were CBOs, the project coordinator aimed to complete 7 interviews with CBOs, or 
30% of 22 agencies. Since 41% of responding agencies were school districts, the 
Project Coordinator aimed to complete 4 interviews with school districts, or 41% of 22 
agencies, and so forth. The following table shows the number of agencies targeted for 
each category type, and the number of interviews actually completed with agencies in 
that category type: 
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Number of 
Interviews 
Targeted 


Number of 

Contacts 

Made 


Number of 
Interviews 
Completed 


Community-based Orgs. 


7 


15 


9 


School Districts 


4 


14 


4 


Literacy Councils 


2 


8 


6 


State Corr. Insts. 


2 


5 


2 


Intermediate Units 


2 


4 


3 


Community Colleges 


1 


1 


1 


Libraries 


1 


4 


2 


Vo-tech Schools 


1 


4 


1 


Universities 


1 


3 


1 


Total 


21 


58 


29 




The Project Coordinator contacted a sampling of agencies across the state (east, 
central, and west) within each category type. She also contacted some agencies who 
indicated on the survey use of instruments for reporting learner that were less common 
but that may have been of potential use to other agencies for their student gains 
reporting (e.g., SLEP, AMES, or WBST). 

The telephone interview consisted of a set of structured questions directed to the 
individual who had completed the mail survey (usually an administrator) or to an 
individual recommended by the person who had completed the survey (for example, a 
teacher who administered or interpreted the results of an assessment). The questions 
asked, for each instrument indicated on the survey as one being used to report learner 
gains: 
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1. why /how the instrument was picked; 

2. how the instrument is working for the agency; and 

3. successes and problems or gaps with the instrument. 

A final question asked the interviewee to identify any assessment needs they have for 
which they do not have an appropriate instrument. It was thought that these questions 
would identify why agencies are using particular instruments for reporting, the qualities 
they look for in selecting instruments for reporting, and gaps or areas where 
instruments are needed. 

The following table shows the number and percentage of the sample providing a 
particular type of service: 





Number of 
Agencies Providing 


Percentage of 
Agencies Providing 


Basic Literacy 


18 


62 


Adult Basic Education 


26 


90 


General Equivalency 
Diploma Level 


23 


79 


English as a Second 
Language 


18 


62 


Work-focus 


6 


21 


Family Literacy 


4 


14 



The following table shows the number and percentage of the sample using a 
particular assessment instrument to report learner gains: 
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Number of Percentage of 

Agencies Using Agencies Using 



Official GED Practice Tests 


13 


45 


GED Tests 


8 


28 


TABE Survey 


9 


31 


TABE Complete Battery 


9 


31 


BEST 


12 


41 


ABLE 


3 


10 


CASAS 


3 


10 


WRMT 


4 


14 


WBST 


2 


7 


SLEP 


2 


7 


AMES 


2 


7 


WRAT 


1 


3 


READ 


1 


3 



The information on the instruments being used by the telephone interview 
sample to report learner gains shows that the profile of the sample is very similar to 
that of the profile of the mail survey respondents: widespread use of the Official GED 
Practice Tests and GED Tests, the two editions of the TABE, and the BEST. Use of 
the more “exotic” instruments is over-represented, due to the fact that the Project 
Coordinator specifically contacted agencies using these tests. 

How instruments used for reporting learner gains were selected 
by agencies. The most commonly reported reason for selecting an instrument across 
the board was that it was accepted for state reporting of learner gains. In the case of the 
TABE editions, particularly, some interviewees reported long-time use of the 
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instrument, satisfaction with it (at least for the most part), and the fact that it met Bureau 
requirements for an instrument used for reporting learner gains. Many interviewees 
also indicated that an instrument was picked because, upon examination, it looked like 
it would fulfill needed functions. Reasons less often cited included that it was 
recommended by another agency working with a similar audience and fulfilling a 
similar purpose, or that it was user-friendly and student-friendly (those using ABLE 
were likely to mention the student-friendliness of this instrument). Cost was rarely 
mentioned as a reason for selecting and instrument, although it was sometimes 
mentioned as a reason why only one instrument was used throughout the agency for 
reporting learner gains. 

How instruments used for reporting learner gains are working: 
Successes, problems, and gaps. It should be noted that many of the contacted 
agencies serving literacy and ESL learners have only begun using any standardized 
assessment in response to the Bureau’s requirement to do so, so some interviewees 
have only been working with their instruments for one to two years. Most agencies 
are supplementing use of their primary instrument or instruments for reporting learner 
gains with other standardized or informal assessments, indicating that seldom does one 
(or even a pair) of standardized assessments show all learners needs or gains. This is 
an extremely positive finding, indicating use of good testing practices by responding 
agencies. 

For contacted agencies serving ABE and GED level learners, the TABE 
Complete Battery Edition, Survey Edition, the ABLE, and the combination of the GED 
Practice Tests/GED Tests appear to be serving needs regarding reporting learner gains 
and providing at least minimal prescriptive information for guiding and planning 
instruction. The Official GED Practice Tests were thought by most contacted agencies 
to be a good predictor of a student’s performance on the GED Tests. However, a few 
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contacted agencies working with GED-Ievel learners indicated that they felt the Official 
GED Practice Tests were not a good predictor of performance on the GED Tests, and 
expressed concern that often scores would decrease from the Practice Tests to the GED 
Tests (this may occur even if the GED Tests were passed). This is a concern for 
agencies anxious to show learner gains. It is beyond the scope of this project to 
speculate as to why this might be happening in some agencies. Many contacted 
agencies working with very beginning literacy learners on reading skills expressed 
concern that the TABE editions, particularly, but to some extent the ABLE as well, are 
not appropriate indicators of these students’ needs nor able to show gains made by 
these learners. This is a problem as well for agencies serving adult learners in other 
states (Alamprese, 1998). Even the new Level L version of the TABE, which is 
designed to measure pre-reading and reading skills of these lowest level learners, was 
thought by some (but not all) of these interviewees to be inadequate in diagnosing 
needs or indicating gains for these learners. One frustration may be caused by the fact 
that the Pre-Reading Skills subtest of Level L (which covers matching letters, 
recognizing letters, and recognizing beginning, middle, and ending sounds) does not 
yield a reportable score for tracking learner gains (raw scores only, not scale scores nor 
grade equivalents, can be obtained from this section). It should be noted, however, 
that TABE and ABLE were thought by others to do a fine job for these beginning-level 
learners. The WRMT was described enthusiastically by three of the four interviewees 
using it as an instrument that works well with basic-level readers. The AMES was 
mentioned by two interviewees as something they intended to look into for use with 
these learners; unfortunately neither of the two agencies who were currendy using 
AMES in the phone sample were using it with basic-level readers, and thus could not 
comment on its usefulness with this population. 
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Two interviewed agencies (school districts) using the WRAT either currently or 
in the past with ABE- and GED-level learners felt that it provided little useful 
information for either diagnosing needs or tracking learner gains. 

Several contacted agencies who are working with a work-focus population in an 
ABLE setting (particularly those who are focusing on a specific career area, such as 
nursing, rather than general employment-related literacy skills) also indicated that the 
TABE editions and the Practice Tests/GED Tests do not identify these students’ needs 
nor show gains made by these learners. The WBST was briefly explored in 
conversations with two of the interview respondents as an instrument that might be 
appropriate for use with this population; however, the two agencies using it had limited 
experience with it. One mentioned that it is not an appropriate instrument for use with 
low-level learners; the other felt that it had not been what she had expected, mostly due, 
she felt, to lack of training in appropriate use of the instrument. 

For contacted agencies serving the ESL population, the BEST is generally 
viewed favorably in terms of reporting learner gains and providing at least minimal 
prescriptive information for guiding and planning instruction. The general exception to 
this is for the upper-level ESL student, greater numbers of whom are increasingly being 
seen in ABLE-funded programs. The SLEP was explored during the course of the 
interviews with the two agencies who are using it as a potentially useful instrument 
with this group of learners. However, those agencies gave the SLEP a mixed review. 
One interviewee indicated that questions seemed difficult and did not appear to relate to 
everyday life; also, the instrument did not appear to adequately measure learner gains. 
The other interviewee felt that it provided a good measure of listening comprehension. 

Summary. While the sample of agencies who participated in the telephone 
interview are generally satisfied with the instruments they are using for reporting 
learner gains, especially when supplemented by other measures for diagnosis of needs 
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and demonstration of gains, three areas where appropriate instruments are needed were 
identified: 

1. Instruments that will better detect instructional needs and educational gain of 
beginning-level adult readers in ABLE programs; 

2. Instruments that will better detect instructional needs and educational gain of learners 
focusing on a specific career area, such as nursing (rather than general employment- 
related literacy skills); 

3. Instruments that will better detect instructional needs and educational gain of upper- 
level ESL learners who have progressed beyond the functional level measured by the 
BEST. 

The need for instruments for beginning-level readers and upper-level ESL has also been 
brought to the attention of project staff by Bill Murphy of the ABLE Bureau. 

Objective 3: To gather information on standardized assessments that 
meet the Bureau’s requirements and that may be appropriate for 
populations served by ABLE programs, including literacy, ABE, GED, 
ESL, and work-focus. 

In November, 1998, the project staff contacted Cheryl Keenan, Sonny Sloan, 
and Ella Morin of the ABLE Bureau for input regarding whether it would be fruitful to 
contact states known to be doing work with the populations named in the project 
proposal. The purpose of the state contacts would be to ask which standardized 
assessments their funded service-providing agencies use for reporting learner gains and 
what criteria or restrictions, if any, are placed on the standardized instruments their 
agencies may use, in the hopes of identifying candidate instruments for use in 
Pennsylvania that are not currently well-known in Pennsylvania. Our Bureau leaders, 
along with Judy Alamprese of Abt Associates, advised that most of the states are using 
the same generalized literacy instruments and standardized instruments targeted to ESL 
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learners to measure learner gains as those are currently used in Pennsylvania, including 
ABLE, TABE, CASAS assessments, and BEST. Therefore, they believed that such 
contacts would probably not gain much useful information. However, Ms. Alamprese 
noted that Abt Associates had completed a review of standardized reading instruments 
that are appropriate for use with first-level adults in programs participating in their 
national study of gains made by first-level adults participating in basic skills instruction. 
Abt Associates had chosen a combination of subtests of the Nelson Reading Skills Test 
(Word Analysis, Fluency, Vocabulary, and Comprehension), the WRAT-III (Reading - 
- i.e. word recognition — and Spelling), and the Woodcock-Johnson Revised Tests of 
Achievement (Background Knowledge and Word Attack) for assessing the gains of 
these learners in their project (Voight & Alamprese, 1998). These subtests were 
selected because they are standardized instruments that have been normed using large 
stratified samples; show evidence of reliability and validity; in combination, measure 
seven constructs that reflect strategies that successful readers have mastered (word 
analysis; isolated word recognition; vocabulary; spelling; comprehension; fluency; and 
background knowledge); have simple requirements for administration; and can be 
administered quickly (in less than 35 minutes). The authors did note that the Nelson 
Reading Skills Test provides norms only on children. Project staff collected 
information on and samples of these assessments to examine them and determine 
whether they may be of use to and useable by program staff in reporting learning gains 
of basic-level readers. 

The Project Coordinator utilized several other strategies to identify instruments 
that may be appropriate for populations served by ABLE programs in addition to those 
that had been identified through the research phase of the project. The focus was 
geared especially toward identifying instruments that would address the needs identified 
by the research phase of the project and Bill Murphy: instruments for basic-level 
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readers, instruments for upper-level ESL learners, and instruments for career-focused 
individuals. These strategies included 1) noting instruments that were being asked 
about on the Project EQUAL listserve, in questions directed to the Project Coordinator 
in her role as a statewide resource person on assessment issues, and during the 
assessment module training sessions conducted throughout the state; 2) noting 
instruments that were recommended by Bill Murphy for review as he worked with 
agencies around their assessment and reporting issues; 3) reviewing the adult education 
literature; 4) reviewing publisher’s assessment catalogs; and 5) keeping abreast of 
developments with the CareerLinks Centers and assessment issues that may arise from 
ABLE-funded agencies’ coordination with these Centers. Through these strategies, a 
variety of instruments was identified for further review. All six standardized but lesser- 
used instruments that ABLE-funded programs are using to report learner gains (see 
Objective 1) were included in the review: WRAT-III, WBST, SLEP, WRMT-R, 
AMES, and SORT-R. 

Instruments for basic-level readers. In addition to the combination of 
subtests identified by Abt Associates, the instruments identified for further review for 
their possible usefulness for basic-level readers included: WRMT-R (a standardized 
instrument identified by agencies participating in the telephone interview as being useful 
for basic-level readers); AMES (another instrument identified by agencies participating 
in the telephone interview as being potentially useful with basic-level readers; and the 
Gates-MacGinitie Reading Test (suggested by Bill Murphy as an instrument being 
used by literacy councils for reporting educational gains that may be useful for the 
beginning reader population). 

Instruments for upper-level ESL learners. The instruments identified 
for further review for their possible usefulness with upper-level ESL learners included: 
the SLEP (a standardized identified through the project research); the BESTEL 
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(identified by Bill Murphy as an instrument that at least one agency was using for 
reporting learner gains and speaking of favorably); and the Adult Language Assessment 
Scales (LAS) published by CTB/McGraw-Hill. 

Instruments for work-focus (especially career-focus) learners. 

The instruments identified for further review for their possible usefulness with work- 
focus/career-focus learners included: the WBST (a standardized instrument identified 
through the project research), Work Keys (being discussed for possible inclusion in the 
CareerLinks Centers assessment battery), and the TABE Work-Related Foundation 
Skills instrument (identified by Adult Education and Job Training Center staff as 
potentially useful for their workplace literacy program). It should be noted that the 
CASAS Employability Competency System provides a criterion-referenced, 
competency-based tool for assessing general pre-employment competencies, although it 
would not address career-specific needs. 

Other instruments selected for further review. Four additional 
instruments were selected for review to see whether they met the Bureau’s requirements 
for instruments appropriate for use with ABLE learners: the SORT-R and the WRAT- 
III (identified through results of the mail survey as standardized instruments that are 
being used by some agencies to report learner gains); the Basic Occupational Literacy 
Test (BOLT) (identified when two programs who wish to use it contacted the Project 
Coordinator to discuss the appropriateness of the test with a work- or life-skills focused 
ABLE populations); and the Test of Applied Literacy Skills (TALS), a standardized test 
for adults publicized by Educational Testing Service but not currendy used in 
Pennsylvania ABLE programs according to the survey. 

Information on these instruments (including descriptions of how the test was 
developed and standardized, information on the norming group, information on 
reliability and validity, and samples of the instrument and directions for administration) 
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was collected from the publishers of the instruments during the second half of the 
project 

Objective 4: To examine potential instruments, and those that are being 
used in agencies, to determine their appropriateness of use related to the 
Bureau’s requirements. 

All instruments were examined to determine whether they meet the Bureau’s 
requirements for instruments to be used for reporting learner gains, in terms of being 
standardized on an appropriate population and having at least two equivalent forms for 
use in pre-testing and post-testing. Also, ease of use by ABLE agencies was 
considered; that is, could staff in ABLE programs reasonably be expected to use the 
instrument in the way it was intended to be used given the varied backgrounds of staff 
and volunteers in programs and the demanding context that they work within? Based 
on this review, a recommendation was made regarding whether each instrument should 
be added to or retained on the Bureau’s list of acceptable instruments for reporting 
learner gains. 

The following sections summarize reviews of and provide recommendations 
regarding instruments in four categories: instruments for general ABE/GED 
populations; instruments for basic-level readers; instruments for upper-level ESL 
learners; and instruments for work-focus/career focus learners. 

Instruments for general ABE/GED populations. The SORT-R is a 
standardized, quick screening instrument that uses a word recognition task as a measure 
of reading level. The SORT-R has documented reliability, validity, and is normed on 
adults; however, it does not provide equivalent forms for use in pre- and post-testing. 
The SORT-R is not on the July, 1998 list of acceptable instruments for reporting 
learner gains, and should not be placed on the list. 
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The WRAT-III is intended to measure achievement of the “codes” needed to 
learn the basic skills of reading Getter reading and word decoding), spelling, and 
arithmetic. The manual gives evidence for reliability and validity of the tests, and it 
includes an adult norming group. Two forms (blue and tan) make it possible to use the 
test in a pre-test/post-test context. However, with such narrow content, it is doubtful 
that the WRAT-III provides any real useful information to agencies. With only two 
brief alternative forms, any gains shown from test to test could likely be due to 
increased test familiarity. Two school districts interviewed as part of the follow-up 
survey to this project indicated that the WRAT did not provide useful information for 
tracking needs or reporting learner gains. The WRAT is currently on the July 1998 list 
of acceptable instruments for reporting learner gains; for the reasons noted above, it 
should be removed from the list 

The AMES is an instrument that has been used by a few agencies for reporting 
learner gains to the state. It is normed on adults; provides evidence of reliability and 
validity; and consists of several levels each with two alternative forms. Level A, the 
lowest level, is intended for use with adults with 1-2 years of schooling or those who 
may be limited-English-proficient or ESL learners. It measures pre-reading and reading 
skills including knowledge of beginning, ending, and middle sounds; reading simple 
signs and symbols; simple vocabulary; and sentence comprehension. A second section 
measures beginning applied math problem solving. Levels B through E are designed 
for adults who have completed varying levels of schooling, from 3 to 4 years for Level 
B through 9 or more years for Level E. Levels B through E measure reading (passage 
and document comprehension) communication (writing conventions, use of a 
dictionary or thesaurus, filling out forms) computation, and applied problem solving. 
The tests appear to have more adult-focused content, with items looking more real- 
world, than those of some other instruments used with school populations. A locator 
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instrument is used to place students into one of the levels. Levels B through E are 
equated onto one scale, and yield raw scores, percentile ranks, stanines, and grade 
equivalents. Level A yields only raw scores, percentile ranks, and stanines; no grade 
level equivalents can be obtained. The Total Level A scores combines both the reading 
and math subtests; a total reading level score alone cannot be calculated. “AMES 
Complete Battery” and “AMES Locator” are included on the July 1998 list of 
instruments acceptable for reporting learner gains. Given the function of the locator 
instrument, it should not be allowed for reporting learner gains. The AMES Complete 
Battery should be kept on the list, however. 

TALS was not indicated by any agencies responding to the survey as an 
instrument that is currently being used to report learner gains. The TALS was 
developed by Educational Testing Service and is a refinement of the instruments used in 
three national large-scale assessments of adult literacy: the NAEP Young Adult 
Literacy survey, the Workplace Literacy Assessment, and the National Adult Literacy 
Survey. The tests take a “profile” approach to measuring literacy that is different from 
the traditional academic skills approach or a competency based approach. The Young 
Adult Literacy Survey yielded evidence for three distinct aspects of literacy: prose, 
document, and quantitative. The tests measure each of these three profiles through two 
forms, A or B. Field-testing and norming of the tests was completed with adults. 
Evidence for reliability and some evidence of validity is provided. Subjects perform the 
tasks in the test booklets and mark their answers directly in the booklet. Tests yield a 
proficiency score for each profile on a scale of approximately 100 - 500. No grade 
level equivalents are given. According to the test manual, the tests may be used for 
estimating proficiency levels, providing planning information and selection of 
materials, and measuring growth and progress; however, little guidance is given about 
how to do this. The tests were field-tested using an adult population. The tests seem 
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as though they would be too difficult for beginning-level learners in reading or math. 
TALS is on the July, 1998 list of instruments acceptable for reporting learner gains and 
would do no harm by remaining on the list. It is difficult to see how agencies would 
use the instrument; however, as it does meet the requirements for a test acceptable for 
standardized reporting, it may provide a useful function for some agencies and thus 
should remain on the list if possible. 

Instruments for basic-level readers. The survey results coupled with the 
follow-up telephone interview results indicated that some agencies are finding that 
frequently-used tests of general literacy skills, i.e. TABE and ABLE, are working to 
document evidence of learning gains and provide guidance for instructional 
interventions with beginning-level readers. The new TABE Level L introduced with 
TABE 7&8 measures pre-reading skills, including matching letters, recognizing letters, 
and knowledge of beginning, ending, and middle sounds; reading skills measured at 
this level include recognizing signs, word meaning, and context, phrase, and sentence 
meaning. Using this level for student reporting is hampered, however, because, as 
previously noted, the pre-reading section yields only raw scores. While ABLE Level 1 
measures the constructs of beginning vocabulary, simple passage/document 
comprehension, and spelling rather than “pre-reading” skills, it was mentioned a couple 
of times specifically during the follow-up telephone interviews as being “student- 
friendly.” ABLE does meet the Bureau’s requirements for instruments acceptable for 
student reporting. Clearly, however, the results of the research showed that these 
instruments are not meeting the needs of ABLE agencies for detecting instructional 
needs or documenting educational gain of these learners. 

The CASAS, which is just beginning to be adopted by agencies in 
Pennsylvania, does offer some options for assessment of basic-level readers through 
the Life Skills System’s Beginning Literacy Reading assessments (Forms 27 and 28) 
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and potentially through the Tests for Developmental Disabilities, Levels AA through 
AAAA (the Tests for Developmental Disabilities are not designed for use with ESL 
learners or those with learning disabilities). The CASAS takes a competency-based 
approach that measures basic skills through life skills; this approach is different from 
the traditional view of reading as a set of basic skills, including word analysis, word 
recognition, vocabulary, comprehension, etc., measured by many tests of beginning- 
level reading. However, for some Pennsylvania agencies whose philosophies and 
instructional methods provide a good match to the competency approach, the CASAS 
instruments could provide some sound options for identifying needs and documenting 
gains of basic-level readers. The Life Skills System was introduced in Pennsylvania 
during the last project year through the basic-level (Initial Implementation Training) 
provided by the Training Module Project. The Tests for Developmental Disabilities are 
not covered through this basic-level training, but the potential of bringing training in 
this system to Pennsylvania as a way of addressing the needs of agencies working with 
basic-level readers should be explored. 

AMES was noted in the follow-up telephone interviews as an instrument that 
was attractive to some respondents as an option to be explored for use with beginning- 
level readers. Unfortunately, Level A provides only a combined reading and math total 
score; the fact that a total reading score alone cannot be calculated lessens its potential as 
an instrument that would be useful to track reading gains of beginning-level readers. 

As previously noted, a study by Abt Associates (Voight & Alamprese, 1998) 
identified a set of instruments to be used for documenting learning gains of first-level 
readers in their national study. These included subtests of the Nelson Reading Skills 
Test (Word Analysis, Fluency, Vocabulary, and Comprehension); the WRAT-III 
(Reading -- i.e., Word Recognition, and Spelling); and the Woodcock-Johnson 
Revised Tests of Achievement (Background Knowledge and Word Attack). The 
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researchers chose this combination of instruments because together they provided 
measures of seven constructs important to skilled reading. 

The Nelson Reading Skills Test measures word meaning (vocabulary) and 
passage comprehension at three levels: A, B, and C. Level A includes a measure of 
word part knowledge (sound/symbol correspondence, root word identification, and 
syllabication), while Levels B and C include a measure of reading rate. The test is 
intended to be used with school children in grades three through nine, and the tests 
were normed on third through ninth graders. Teacher judgment in conjunction with 
prior test scores is used to assign students to appropriate levels; no locator instrument 
exists. Evidence for reliability and face validity is provided, and two forms (3 and 4) 
exist. Grade equivalents may be obtained for the word meaning, comprehension, and 
reading rate subtests, but not for the word parts subtests. Total Reading raw scores and 
grade equivalents may be obtained by combining scores of the word meaning and 
reading comprehension subtests; but the Total Reading Score does not include scores 
from the word parts or reading rate subtests. The Project Coordinator learned, via a 
telephone call with Riverside Publishing Company, publishers of the Nelson Reading 
Skills Test, that the test is being discontinued; no new materials for the current Forms 3 
and 4 are being printed and they will only be available until current stock runs out 
This advises against promoting any widespread use of the instrument at this point, 
whether used alone or in conjunction with other instruments. “Nelson Reading” does 
appear on the July, 1998 list of tests acceptable for ABLE reporting; however, given 
the fact that the test is being phased out, plus the fact that it is normed on a child rather 
than adult population, it is probably not advisable even for agencies who may still have 
materials to use it for reporting learner gains and should thus be removed from the list. 

The Woodcock-Johnson Revised Tests of Achievement are part of the 
Woodcock-Johnson Psychoeducational Battery, a set of individually-administered tests 
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“for measuring cognitive abilities, scholastic aptitude, and achievement.” The tests may 
be administered for assessing educational needs and growth. The tests were normed on 
an adult sample, provide evidence for reliability and validity, and have two available 
forms (A and B). Suggested “Starting Points” in each subtest (based on an estimate by 
the administrator of the subject’s achievement by grade level, “average adult,” or 
“above-average adult”) are used to determine which item to begin administration with; 
basal and ceiling levels are used to reduce testing time. There is a standard battery and a 
supplemental battery; across these two batteries, several subtests measure reading 
skills, including: Letter/Word Identification (standard battery); Passage 
Comprehension” (standard battery); Word Attack (supplemental battery); Vocabulary- 
Synonyms (supplemental battery); and Vocabulary- Antony ms (supplemental battery). 
The standard battery also includes three “knowledge” subtests — Science, Social 
Studies, and Humanities — that Abt Associates used to measure adult beginning 
readers’ general background knowledge. Developing readers’ abilities may be 
hampered by lack of academic knowledge that was not acquired over their lifetimes due 
to poor reading skills. Having a simple measure of background knowledge is an 
attractive aspect of these tests. The tests yield raw scores, scaled scores, and grade 
equivalents for each individual subtests and for test clusters including broad reading, 
basic reading skills, reading comprehension, and broad knowledge; one of the reading 
clusters could be chosen for LiteracyPro reporting. These tests grew out of a tradition 
of one-on-one diagnostic assessment of special needs learners. The tests appear as 
though they would be relatively simple for ABLE staff to learn to administer; however, 
the manuals are quite daunting and relevant information on scoring and interpretation 
could be difficult to extract. Training would probably be needed in order to promote 
widespread adoption of these tests by ABLE programs. “Woodcock-Johnson 
Psychoeducational Battery” is currently on the list of acceptable reporting instruments; 
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as the instruments are appropriate and could provide an option for measuring needs and 
gains of basic-level readers, this listing should be modified to read “Woodcock- 
Johnson Revised Tests of Achievement” and the achievement tests should be included 
on the list. 

The WRMT-R was also developed by Richard Woodcock, one of the 
developers of the Woodcock- Johnson Revised Tests of Achievement. It is an 
individually-administered instrument similar in concept and format to those tests. It 
includes a battery of tests that measure important aspects of reading very similar to 
those on the Tests of Achievement. The manual provides evidence of reliability and 
validity and adult norms. Two forms of the tests (G and H) are available. Subtests 
measure the following skills: visual/auditory learning (Form G only); letter 
identification (Form G only); word identification (word recognition); word attack; word 
comprehension (vocabulary); and passage comprehension. Raw scores, scaled scores, 
and grade equivalents are available for individual subtests, “clusters” of subtests (e.g., 
readiness, basic skills, and reading comprehension), plus an overall total reading 
cluster. ABLE programs could not report scores in the readiness cluster, because it is 
measured only by Form G. These tests provide a potentially useful measure for 
reading skills of basic-level readers, but, for the same reasons mentioned regarding the 
Woodcock-Johnson Revised Tests of Achievement, would most likely require training 
for programs interested in using it. The Woodcock Reading Mastery Test-Revised 
version should remain on the list of acceptable instruments for reporting. 

The Gates-MacGinitie Reading Tests are designed to indicated general reading 
achievement The manual gives evidence of the tests’ reliability and validity. The tests 
were normed on kindergarten through 12th graders. The tests consist of several levels, 
some having one form while others have two forms. The levels include: Pre, which 
measures literacy concepts, reading instruction relational concepts, oral language 
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concepts, and letters and letter-sound correspondences (via Form K only); R, which 
measures use of letter/sound correspondences and use of sentence context (also via 
Form K only); 1, which measures vocabulary and passage comprehension (again via 
Form K only); and 2 through 10/12, which measure vocabulary and passage 
comprehension (each level via 2 forms — K and L). Each level is designed to be given 
to students in that corresponding grade in school; it is difficult to see how the 
instrument would be used with adults because there is no locator procedure, and 
teachers would essentially need to know the grade level the adult is operating at in order 
to assign a level. The content of the test is very academically-focused and not geared 
toward adults in the way that TABE, ABLE, or the CASAS instruments are. The 
Project Coordinator learned, via a phone call with Riverside Publishing Company, that 
a new version of the Gates-MacGinitie Reading Tests including an adult level will be 
available after September, 1999. The Gates-MacGinitie Reading Tests are on the July, 
1998 list of instruments acceptable to be used for reporting purposes, but should be 
removed due to the tests’ lack of adult norms, lack of adult-focused content, and the 
difficulty of selecting a level to use with an adult learner. However, the new version of 
the tests should be examined as soon as they are available to assess their potential for 
use in tracking the learning gains of basic-level adult readers. 

Instruments for upper-level ESL learners. The CASAS offers some 
options for assessment of upper-level ESL adults through the Life Skills System and 
the Employability Competency System. These CASAS systems offer competency- 
based reading and listening comprehension subtests across a range of ability levels 
wider than the those of the BEST. Both of these systems were introduced through the 
Initial Implementation Training in Pennsylvania last year. As more agencies adopt 
these CASAS systems, they may find that the systems work for these upper-level ESL 
learners. 
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The SLEP measures English language ability in listening comprehension and 
reading comprehension. Section 1 of the test uses recorded samples of spoken English 
to test listening comprehension. Section 2 measures vocabulary, grammar, and reading 
comprehension through written and visual materials. Currendy, three forms of the 
SLEP (1, 2, and 3) are available. It is designed for use with students entering school 
grades seven through twelve whose native language is other than English, and that was 
the norming group for the test. The test manual provides evidence for reliability and 
validity. The test materials utilize situations typical of those encountered in North 
American secondary schools. The SLEP is on the July, 1998 ABLE list of instruments 
acceptable for reporting learning gains. Despite the fact that the test is not normed on 
adults, it should be retained on the list. The SLEP is a sound test that provides one of 
the few options available for measurement of basic listening and comprehension 
reading skills outside of a competency-based approach. Just as the Official GED 
Practice Tests are normed on groups of high-school seniors and are acceptable as 
measures of adults’ attainment of this body of knowledge, so the SLEP can be seen as 
a test that is worthy of measuring adults’ attainment of secondary-level English 
proficiency. ABLE agencies using the test may be cautioned to confirm its usefulness 
with their students. Adults from other cultures are not be familiar with American 
customs and society may be at a disadvantage in performing on the test. 

The Adult Language Achievement Scales (LAS) consists of oral language, 
writing, reading, and math components. The reading and math components have two 
levels. Performance on the oral language component is used to assign students to the 
appropriate reading and math levels. Two forms (A and B) of each component are 
available. The oral language component is administered one-on-one and both the oral 
language and writing components utilize holistic scoring. The test manual provides a 
training exercise for establishing inter-rater reliability in the scoring procedures. The 
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Adult LAS is the latest iteration in a series of tests that have been in development since 
the late 1970s. The tests do not provide a standard score scale, and so are not 
“standardized” tests. The Adult LAS may provide programs with some useful informal 
assessment information, but at this time the Adult LAS should not be added to the list 
of instruments acceptable for reporting learner gains. 

The BESTEL is a revision of the STEL forms listed on the July, 1998 ABLE 
list of acceptable tests for reporting. It is published by the Association of Classroom 
Teacher Testers. The Project Coordinator tried to contact the Association several times 
at their last known mailing address in Montecito, CA, through their telephone number, 
and through their web site. She was unable to contact anyone in order to request 
further information or materials for review. Thus, at this point, this project cannot 
address whether the STEL/BESTEL should be retained on the list or removed. 

Instruments for work-focus/career-specific learners. Work Keys 
assessments are part of the Work Keys system developed by ACT for documenting and 
improving workplace skills. The Work Keys system provides, among other services, 
opportunities for specific job profiling within a company. Work Keys assessments 
provide an avenue for measuring individuals’ skill levels to determine how they 
compare to the skills needed by specific jobs. Eight criterion-referenced assessments 
exist, including listening, reading for information, writing, applied mathematics, 
applied technology, locating information, observation, and teamwork. Because the 
tests are highly secure, the Project Coordinator was unable to obtain actual samples of 
the tests or detailed technical information about them. However, according to 
descriptions published by ACT, they appear to yield scores on a broad scale of 3 - 7. 
The fact that the score scale is so broad indicates that it would most likely not be useful 
for meeting ABLE’s needs in tracking learner gains. It is not clear that at least two 
forms of each of the tests exist. It is not clear whether Work Keys would be 
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appropriate for career-specific audiences as well as work-focus audiences. It does not 
appear that Work Keys should currently be added to the list of acceptable reporting 
instruments. However, if it is adopted by the CareerLinks system more information on 
the assessments may become available, and if it does, a reassessment of the instruments 
should be done at that time. 

The WBST provides a relatively brief but psychometrically sound measure of 
basic reading and math skills through two subtests: verbal and quantitative. Good 
evidence for reliability and validity is provided by the user’s manual. The tests were 
normed on an adult population and two forms of each subtest (1 and 2) are provided. 
The verbal skills test assesses students’ abilities to locate, understand, and use 
information in various formats, including maps, charts, and other written information; 
recognize word meanings in context; and recognize and identify proper grammar and 
sentence construction. The quantitative skills test measures basic math computation, 
quantitative evaluation, algebra, and geometry. Scoring can be done only with a 
computer disk provided by Wonderlic Incorporated, developers of the test. It appears 
that test data must be manually entered into the computer. Results of the WBST can be 
compared to the basic skills requirements of various occupations listed in the U. S. 
Department of Labor’s “Dictionary of Occupational Titles.” The “Dictionary of 
Occupational Titles” lists the math and language skill requirements of jobs in terms of 
the six-level General Educational Development (GED) Scale. The WBST measures 
skills at the lowest three GED Scale levels, and provides a score on a scale of 1 to 500 
for verbal and quantitative skills at each of the first three GED levels as well as a 
composite score for verbal and quantitative skills. It also provides a Total Verbal Skills 
Score and Total Quantitative Skills Score and grade level equivalents for verbal and 
quantitative skills that could be used for ABLE reporting. Unfortunately, the WBST 
provides grade level equivalents only for grades six through 12, indicating that the 
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tests measure the upper-ABE and GED skill range and that they are most appropriate 
for those populations. The WBST is on the July, 1998 list of acceptable instruments 
for reporting and should remain as a measure of basic skills that can be linked to skill 
needs of a variety of occupations. Clearly it is a measure appropriate for work-focus 
rather than career-specific populations. 

The TABE Work-Related Foundations Skills assessment is essentially an 
extension of TABE 7&8, and as such is not appropriate for career-specific learners. It 
assesses the TABE7&8 objectives at Level D in Trade/Technical, Health, or 
Business/Office contexts; students who need a different level or alternative form of the 
tests are administered another level of TABE 7&8 or forms 7 or 8 of Level D. TABE 
Work-Related Foundation Skills is not included on the July, 1998 list of instruments 
acceptable for reporting learner gains, and should probably not be noted separately in 
future lists as it is an extension of TABE 7&8. 

The BOLT was developed in the 1970s as an assessment of basic literacy skills 
by the U. S. Department of Labor for use in the Manpower programs. The BOLT 
measures basic reading and arithmetic skill at four levels; fundamental, basic 
intermediate, intermediate, and advanced. There are four subtests: reading vocabulary, 
reading comprehension, arithmetic computation, and arithmetic reasoning. These may 
be administered independently. Each subtest has two or three alternative forms at each 
level. The appropriate level of each subtest to administer a student is determined by 
using scores from the Wide Range Scale (a locator test) and the individual’s reported 
years of education. The BOLT yields raw scores, standard scores, and GED Scale 
levels (1-4). The tests were normed on adults. The manual that describes 
development of the tests provides no evidence for validity of the tests. The tests are 
extremely brief and inexpensive, but unfortunately, narrow in content and probably not 
adequate as a measure of learner gains needed by ABLE. BOLT is not on the July, 
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1998 list of instruments acceptable for reporting and should not be included on future 
lists. 

Objective 5: To develop a revised listing of acceptable standardized 
instruments, to be used by agencies for reporting educational gains, for 
approval and dissemination by the Bureau. 

This section summarizes recommendations to the Bureau for refining its list of 
instruments acceptable for reporting learner gains based on the findings of the project 
described to this point. The recommendations are organized to address the needs of the 
four ABLE populations addressed in this research: ABE/GED learners, basic-level 
readers, ESL learners, and work-focus/career-specific learners. 

There are several instruments on the July, 1998 list of instruments acceptable 
for reporting learner gains that are not addressed in this project (for example, API, 
Gilmore Oral Reading Test, Stanford Achievement Test, and others). Response to the 
survey indicated that these instruments may be used by one or two programs if at all. 
Generally, these instruments should probably be removed from the list unless a sound 
case for including them can be made. Each test would need to be reviewed to determine 
whether it meets the Bureau’s requirements for acceptable reporting instruments. 

ABE/GED population. 

1. The SORT-R, while it continues to be used for reporting learner 
gains by some agencies according to the results of the mail survey, does not provide 
equivalent forms. It is not on the July, 1998 list of acceptable reporting instruments 
and should remain off of the list. 

2. The WRAT-III is on the July, 1998 list but should be removed due to 
the fact that it is an extremely brief instrument designed to measure only the “codes” of 
reading, spelling, and math. It appears to provide little useful information regarding the 
Bureau’s requirements for tracking learner gains. 
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3. AMES is a little-used, but sound, adult-focused instrument that 
would be useful for tracking gains of the ABE/GED population. Both the AMES 
locator and Complete Battery are listed on the July, 1998 list of acceptable instrument. 
The intended function of the locator makes it inappropriate for use in reporting learner 
gains to the state, and so the locator should be removed from the list; however, the 
AMES Complete Battery should remain on the list. 

4. The TALS is a sound instrument that provides a unique approach to 
assessing the skill levels of ABE/GED level learners and so should be retained on the 
list of acceptable reporting instruments. 

5. The July, 1998 list of acceptable instruments for reporting includes 
both the TABE locator and ABLE “short form.” These are locator instruments and, as 
such, are not suitable for use in reporting learner gains. These instruments should be 
removed from future lists. 

6. Contemporary’s “Diagnostic Pre-tests for GED Instruction” and 
“Evaluative Post-tests for GED Readiness,” while they are undoubtedly useful for 
identifying student needs regarding preparation and readiness for the GED tests, are 
non-standardized instruments and as such should not be included on the list of 
instruments acceptable for reporting learning gains. Generally, informal curriculum- 
based GED readiness assessments are inappropriate for inclusion on this list. At this 
point, only Steck-Vaughn’s print version of the Official Practice Tests and 
Contemporary’s software version of the Official GED Practice Tests can provide 
standardized data on learner gains and are thus appropriate to include on the list. 

Basic-level readers. 

1. The READ and the Brigance Diagnostic Inventories for adults are 
informal inventories that will be very useful for some agencies in diagnosing specific 
skill needs of beginning readers. However, as informal inventories, they are 
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inappropriate to include on the list of acceptable reporting instruments. The READ and 
the Brigance Diagnostic Inventory of Essential Skills are on the July, 1998 list of 
acceptable reporting instruments, but should be deleted from future lists. 

2. CASAS’ Life Skills System Beginning Literacy Reading assessments 
and Tests for Developmental Disabilities have potential for tracking the gains of 
beginning readers through a competency-based approach. The Life Skills system was 
introduced in Pennsylvania during the last project year; some agencies are beginning to 
adopt the system and its success in meeting the needs of this population should be 
tracked. The Tests for Developmental Disabilities were are not covered in the basic- 
level training introduced in Pennsylvania last year. The potential of this system to meet 
the needs of these learners should be explored. 

3. Use of the combination of instruments proposed by Abt Associates 
for tracking the learning gains of the basic-level readers in their national study (Nelson 
Reading Skills Test [Word Analysis, Fluency, Vocabulary, and Comprehension], the 
WRAT-III [Reading — i.e. word recognition — and Spelling], and the Woodcock- 
Johnson Revised Tests of Achievement [Background Knowledge and Word Attack]) is 
not feasible at this time: the Nelson Reading Skills Test has been discontinued and test 
materials will not be available once current stock runs out. A combination of some 
other instruments that measure at least some of the same constructs could be 
considered. How use of such a combination instruments would be compatible with 
LiteracyPro would need to be explored. 

4. “Nelson Reading” appears on the July, 1998 list of instruments 
acceptable for reporting gains. The Nelson Reading Skills Test is a sound instrument 
that is normed on children rather than adults; thus, it is not advisable for agencies who 
may still have materials to use it in isolation for reporting learner gains. This, coupled 
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with the fact that the test is being phased out, indicates that it should be removed from 
future lists. 

5. The Woodcock-Johnson Revised Tests of Achievement are part of 
the Woodcock-Johnson Psychoeducational Battery which is included on the July, 1998 
version of the reporting list. These tests could provide a one-on-one approach for 
measuring some of the skills and background knowledge of basic-level readers; the 
feasibility of their expanded use with basic-level readers should be explored. The 
complexity of the manual and its guidelines for administering, interpreting, and scoring 
the tests indicate that developing training for program staff around the instrument may 
be prudent. The Woodcock-Johnson Revised Tests of Achievement should be retained 
on future lists of instruments acceptable for reporting. 

6. The WRMT-R is an instrument similar in concept and format to the 
Woodcock-Johnson Revised Tests of Achievement that focuses on reading skills. 
Programs who wanted to adopt these tests would most likely benefit from training in 
proper use of the tests. Subtests measure basic reading skills; results of the subtests 
can be summarized into three clusters, including readiness, basic skills, and reading 
comprehension. ABLE programs could not report scores in the readiness cluster, 
because these subtests are measured through only one form. The feasibility of 
expanded use of this instrument with basic-level readers should also be explored. 
“Woodcock Reading Mastery Test” is included in the July, 1998 and the Revised 
version should be included in future lists. 

7. The Gates-MacGinitie Reading Tests are on the July, 1998 list of 
instruments acceptable for reporting purposes, but should be removed due to the lack of 
adult norms, adult-focused content, and the difficulty of assigning adult learners to an 
appropriate level of the tests (no locator instrument exists). A revised version including 
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an adult level will be available after September, 1999, and should be reviewed to assess 
its potential for tracking needs and gains of basic-level readers. 

ESL learners. 

1. ESLOA is an informal instrument that some interviewees mentioned 
in the follow-up telephone survey as being similar to the BEST and useful for very 
low-level ESL learners with its oral assessment component. It is included on the July, 
1998 list of instruments acceptable for reporting learner gains, but as an informal 
instrument, is not suitable for this purpose. ESLOA should be removed from future 
versions of the list. 

2. CASAS offers some options for upper-level ESL assessment through 
its competency-based Life Skills and Employability Competency Systems. These testes 
measure reading and listening comprehension. Both of these systems were adopted 
through Initial Implementation Training in Pennsylvania last year. As more agencies 
adopt these CASAS systems, they may find them useful for the upper-level ESL 
population. 

3. The SLEP is a sound test that provides one of the few options 
available for measurement of basic listening and comprehension skills outside of 
competency-based approach. SLEP is on the July, 1998 list of acceptable reporting 
instruments and should remain on future lists. 

4. The BESTEL is a revision of the STEL forms included on the July, 
1998 list of instruments acceptable for reporting. Despite numerous attempts to contact 
the Association of Classroom Teacher Testers, developers of the BESTEL, the Project 
Coordinator was unable to obtain further information about the tests or copies for 
review. At this point, the project cannot directly address whether STEL/BESTEL 
should be retained on the list or be removed. Lack of responsiveness on the part of the 
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test developers indicates that this assessment would not provide a realistic option for 
expanded use with ESL learners. 

Work-focus/career-specific learners. 

1. No standardized instruments suitable for tracking the learner gains of 
career-specific learners (e.g., certified nursing assistant trainees) were located during 
the course of this project. 

2. The WBST provides a sound option for tracking basic reading and 
math skills of work-focus learners. The WBST is attractive because its results can be 
compared to basic skill requirements of various occupations listed in the U. S. 
Department of Education’s “Dictionary of Occupational Titles.” The tests are most 
appropriate for upper-ABE and GED populations. The WBST is on the July, 1998 list 
of instruments acceptable for reporting and should remain on future lists. 

3. The BOLT is an extremely brief measure of basic reading and math 
skills developed during the 1970s for use in the Manpower programs. The 
development manual provides no evidence for validity of the tests. The tests are, 
unfortunately, too narrow in content to adequately measure learning gains needed by 
ABLE. The BOLT is not on the July, 1998 list of instruments acceptable for reporting, 
and should not be included on future lists. 

Additional Special Issues 

In June, 1999, Bill Murphy sent to the project staff a list of additional issues the 
Bureau hoped the project would address. This section provides comment on each of 
these issues. 

1. Which tests other than the TABE editions, the ABLE, CASAS, the BEST, 
and the Official GED Practice Tests should be included in the Literacy Pro software 
systems for reporting learner gains? The LiteracyPro software should be adapted to 
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accept score entry from all of the instruments included on the revised list of acceptable 
reporting instruments. 

2. Should matching TABE scores with GED scores (via the “GED Predictor 
Report” published by CTB/McGraw-Hill) be allowed for in the LiteracyPro software? 
Yes. The “GED Predictor Report” allows GED scores to be estimated from TABE 
scale scores. Providing an adaptation to the LiteracyPro software that would allow 
matching TABE scores with GED scores would be a legitimate use of this instrument 
that would greatly assist agencies in their ability to meet the Bureau’s program 
performance standards. 

3. Should tests scores from the subsequent fiscal year by accepted as post-tests 
by the LiteracyPro software? Should GED tests passed after July 1 be credited to the 
previous program year? From an assessment point of view, it doesn’t matter which 
fiscal year the pre- and post-testing occurs in, as long as the post-testing is done after a 
significant amount of instructional hours. Adjustments that support learners and 
programs in their efforts to advance learner goals should be supported by the software. 

4. How much time should elapse before the same form of a test is used again? 
Is six months adequate? How should students who have intensive programs of study 
be tested? Should those agencies go beyond 50 hours of instruction before post-testing 
if those hours are earned within one to two months? Few if any test manuals address 
this issue directly. However, refraining from using the same form of a test for about 
six months appears to be a good rule of thumb. Programs and agencies that provide 
intensive instructional programs, where 50 hours of instruction may be accumulated by 
a student in a matter of a few weeks, should go beyond 50 hours before re-testing 
students if necessary, re-testing approximately every three months. This is in line with 
the current program performance standard which indicates that students should be 
reassessed after at least 50 hours of instruction. It seems that the ideal would be to 



Assessment Practices Final Report p.46 



have the LiteracyPro software allow agencies to enter the number of hours of 
instruction achieved with each re-test (post-test) score (and this may already be done). 
This would allow the Bureau to track gains as they occur over varying amounts of 
instructional time (25 hours, 50 hours, 100 hours, etc.). 

5. Should agencies be told not to enter scores (into the LiteracyPro software) 
which are at the top of the distribution for a test level? Should the LiteracyPro software 
be modified to edit check for that? Agencies should be educated to understand which 
test scores for the instrument they are using are inappropriate for assessing gain or 
providing information for instructional planning, and to identify instances when re- 
assessing a particular learner may be necessary. It seems at this time, however, that 
modifying the LiteracyPro software to edit check for this is not advisable; these 
instances are not always clear and sometimes require a judgment call based on the 
circumstances of the particular student, etc. 

6. Should the Bureau disallow reporting scores of short tests such as the 
SORT, WRAT, BOLT, or GATB? Should the Bureau require individualized sessions 
for learners for test score debriefing? The project provided evidence that the SORT-R, 
the BOLT, and the WRAT-III are not appropriate for meeting the Bureau’s needs in 
terms of providing sound information on student learning gains. The current project 
did not review the GATB. While it seems that encouraging individualized meetings 
with all learners for the purpose of test score debriefing and goal setting would be ideal, 
providing commentary on this issue is beyond the scope of this project 

Evaluation Techniques, Instruments, and Results 

Throughout the project, the project staff worked closely with Bureau staff to 
monitor progress toward project goals. The design of the project ensured that the field 
contributed to identifying standardized assessment instruments that could be used to 
document educational gains of ABLE populations. 
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Procedures for the Dissemination of the Findings and Products 

The Final Report will be disseminated through the Adult Education and Job 
Training Center, the Bureau of Adult Basic and Literacy Education, ABLE’s regional 
Professional Development Centers, and the AdvancE State Literacy Resource Center. 

Conclusions and Recommendations 

This project provided some sound data for assisting the Bureau as it began the 
process of revising its list of instruments acceptable for reporting learner gains. A 
general recommendation to the Bureau is to strive for including as wide a range as 
possible of acceptable instruments for each population to allow agencies choice in 
selecting an instrument that matches their program goals and philosophies and learner 
needs. Clearly, at this point in time, this list is relatively short for each of the relevant 
populations. Each instrument, obviously, will have pros and cons associated with its 
use, and the Bureau should continue to encourage agencies, as it has in the past, to use 
a variety of standardized and informal instruments to obtain the range of assessment 
information they need. In the future, other possibilities in new or revised instruments 
should continue to be explored. 
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ABLE Assessment Instruments 
Survey 



1. Please check all published assessment instruments that you currently use for 
pre- and post-testing to report learner gains to the state: 

TABE Survey 

TABE Complete Battery 

BEST 

ABLE 

CASAS 

Official GED Practice Tests 

GED Tests 

READ (Reading Evaluation Adult Diagnosis) 

ESLOA (English as Second Language Oral Assessment) 

Other(s): (Please include the publisher of each assessment listed.) 



2. Please check all published assessment instruments that you currently use for 
purposes other than to report learner gains to the state: 

TABE Survey 

TABE Complete Battery 

BEST 

ABLE 

CASAS 

Official GED Practice Tests 

GED Tests 

READ (Reading Evaluation Adult Diagnosis) 

ESLOA (English as Second Language Oral Assessment) 

Other(s): (Please include the publisher of each assessment listed.) 



3. Identify your agency as one of the following: 

Literacy Council 

School District 

Intermediate Unit 

Library 

Community-based Organization 

Community College 

Other: 

4. Name of person filling out survey: 

Agency: 

Complete Mailing Address: 



Phone: 



E-mail: 



Appendix B: 

Survey Respondents by Agency Type 



Type of Agency 



Survey Respondents by Agency Type 

Number Indicating Percent Indicating 



Literacy Council 


21 


10 


School District 


41 


19 


Intermediate Unit 


19 


9 


Library 


9 


4 


Community-based Organization 


64 


30 


Community College 


11 


5 


Institution 


1 


.5 


University 


5 


2 


State Correctional Institution 


22 


10 


Vocational -technical School 


6 


3 


College 


2 


1 


Library Literacy Program 


2 


1 


County Prison 


3 


1 


Post-secondary Trade 

and Technical School 


1 


.5 


Community Corrections 


1 


.5 


Alternative High School 


1 


.5 


Non-profit 


2 


1 


Union-based Training 


1 


.5 


Correctional Facility 


1 


.5 


Long-term Health Care Facility 


1 


.5 


Social Service Agency 


1 


.5 


Public Housing 


1 


.5 
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Appendix C: 



Number and Percentage of Each Agency Type Using Instrument 
to Report Learner Gains 



Number and Percentage of Each Agency Type Using Instrument 

to Report Learner Gains 



Instrument 
(see key on 
following page for 
full name and 
publisher) 


Standardized? 


Community-based 
Organization n=64 


School District 
n=41 


Literacy Council 
n=21 


State Correctional 
Institution n=22 


Intermediate Unit 
n=19 


Community College 
n=l 1 


Library 

n=9 


Vocational-technical 
School n= 6 


University 

n=5 


TABE (Survey 


Y 


13 


12 


10 


15 


11 


3 


4 


2 


1 


Edition) 




20% 


29% 


48% 


68% 


58% 


27% 


44% 


33% 


20% 


TABE (Complete 


Y 


15 


12 


6 


19 


8 


5 


1 


2 


4 


Battery Edition) 




23% 


29% 


29% 


86% 


42% 


45% 


11% 


33% 


80% 


BEST 


Y 


18 


7 


14 


2 


4 


4 


2 


0 


2 






28% 


17% 


67% 


9% 


21% 


36% 


22% 




40% 


ABLE 


Y 


10 


3 


2 


0 


3 


1 


0 


1 


0 






16% 


7% 


9% 




16% 


9% 




17% 




CASAS 


* 


5 


1 


2 


0 


5 


2 


0 


0 


0 






8% 


2% 


10% 




26% 


18% 








Official GED 


Y 


28 


32 


14 


15 


11 


8 


4 


6 


2 


Practice Tests 




44% 


78% 


67% 


68% 


58% 


73% 


44% 


100% 


40% 


GED Tests 


Y 


18 


22 


8 


15 


11 


8 


2 


2 


2 






28% 


54% 


38% 


68% 


58% 


73% 


22% 


33% 


40% 


READ 


N 


1 


1 


1 


0 


0 


0 


0 


0 


0 






2% 


2% 


5% 














ESLOA 


N 


4 


1 


3 


0 


1 


1 


1 


0 


1 






6% 


2% 


14% 




5% 


9% 


11% 




20% 


WRAT 


Y 


5 


5 


1 


1 


2 


0 


1 


0 


0 






8% 


12% 


5% 


5% 


10% 




11% 






Brigance 


N 


4 


2 


0 


1 


1 


0 


0 


0 


0 


Assessments 




6% 


5% 




5% 


5% 










WBST 


Y 


2 


0 


1 


0 


1 


0 


0 


0 


0 






3% 




5% 




5% 










SLEP 


Y 


1 


1 


1 


0 


1 


0 


0 


0 


0 






2% 


2% 


5% 




5% 










Contemporary GED 


N 


1 


2 


0 


0 


1 


0 


0 


0 


0 






2% 


5% 






5% 










WRMT 


Y 


1 


0 


2 


0 


0 


0 


1 


0 


0 






2% 




9% 








11% 






AMES 


Y 


1 


0 


0 


0 


0 


0 


1 


0 


1 






2% 












11% 




20% 


SORT 


Y 


1 


0 


0 


0 


0 


0 


2 


0 


0 






2% 












22% 







* most tests are standardized 
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Key to Test Abbreviations and Publishers 



TABE (Survey) 


Tests of Adult Basic 
Education (Survey) 


CTB/McGraw-Hill 


TABE (Complete Battery 


Tests of Adult Basic 
Education (Complete 
Battery) 


CTB/McGraw-Hill 


BEST 


Basic English Skills Test 


Center for Applied 
Linguistics 


ABLE 


Adult Basic Learning 
Examination 


Harcourt Brace 


CASAS 


Comprehensive Adult 
Student Assessment 
System 


CASAS 


Official GED Practice Tests 


Official GED Practice Tests 


Steck-Vaughn 


GED Tests 


GED Tests 


GED Testing Service 


READ 


Reading Evaluation Adult 
Diagnosis 


Literacy Volunteers of 
America 


ESLOA 


English as a Second 
Language Oral Assessment 


Literacy Volunteers of 
America 


WRAT 


Wide Range Achivement 
Test 


Wide Range, Inc. 


Brigance Assessments 


Brigance Assessments 


Curriculum Associates 


WBST 


Wonderlic Basic Skills 
Test 


Wonderlic, Inc. 


SLEP 


Secondary Level English 
Proficiency Test 


Educational Testing 
Service 


Contemporary GED 


Contemporary GED 


Contemporary 


WRMT 


Woodcock Reading 
Mastery Test 


AGS 


AMES 


Adult Measure of Essential 
Skills 


Steck-Vaughn 


SORT 


Slosson Oral Reading Test 


Slosson Educational 
Publications, Inc. 


TABE W-F 


Tests of Adult Basic 
Education Work-Related 
Foundation Skills 


CTB/McGraw-Hill 


API 


Adult Performance 
Indicator 


Hadley Press 
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Other Instruments Used to Report Learner Gains 



Botel Word Opposite Test (Follett) 

TABE Work-Related Foundation Skills 
API (Hadley Press) 

MELT 

Jerry Johns Basic Reading Inventory (Kendall-Hunt) 
GALS 

Steck-Vaughn GED 2001 

Slosson 

Woodcock 

RAT 

National Service Center (NSC) Test 
TASK 

Gates-MacGinitie 
Stanford Diagnostic Math 
Laubach Placement/Diagnostic 
Side by Side 

Gilmore Oral Reading Test (Harcourt Brace Jovanovich) 

Stanford Diagnostic Reading Test 

Gilmore 

Peabody 

Iowa 
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Number and Percentage of Each Agency Type Using Instrument 
for Other Purposes 
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Number and Percentage of Each Agency Type Using Instrument 

for Other Purposes 



Instrument 
(see key in 
Appendix C for full 
name and 
publisher) 


Community-based 
Organization n=64 


School District 
n=41 


Literacy Council 
n=21 


State Correctional 
Institution n=22 


Intermediate Unit 
n=19 


Community College 
n=l 1 


Library n=9 


V ocational-technical 
School n=6 


University 

n=5 


TABE (Survey 


8 


9 


7 


19 


8 


3 


2 


1 


0 


Edition) 


13% 


22% 


33% 


86% 


42% 


27% 


22% 


17% 




TABE (Complete 


8 


5 


4 


15 


5 


5 


0 


3 


1 


Battery Edition) 


13% 


12% 


19% 


68% 


26% 


45% 




50% 


20% 


BEST 


5 


3 


4 


0 


1 


1 


2 


0 


0 




8% 


7% 


19% 




5% 


9% 


22% 






ABLE 


7 


0 


0 


0 


0 


0 


0 


1 


0 




11% 














17% 




CASAS 


3 


1 


0 


0 


3 


2 


0 


0 


0 




5% 


2% 






16% 


18% 








Official GED 


35 


20 


8 


21 


12 


5 


4 


3 


3 


Practice Tests 


55% 


49% 


38% 


95% 


63% 


45% 


44% 


50% 


60% 


GED Tests 


20 


11 


6 


20 


10 


4 


3 


1 


0 




31% 


27% 


29% 


91% 


53% 


36% 


33% 


17% 




READ 


2 


1 


1 


0 


0 


0 


1 


0 


0 




3% 


2% 


5% 








11% 






ESLOA 


2 


1 


4 


0 


0 


1 


1 , 


0 


0 




3% 


2% 


19% 






9% 


11% 






SORT 


4 


2 


2 


2 


0 


0 


2 


0 


1 




6% 


5% 


9% 


9% 






22% 




20% 


WRAT 


1 


5 


0 


2 


2 


1 


1 


0 


0 




2% 


12% 




9% 


11% 


9% 


11% 






Contemporary 


3 


1 


0 


0 


0 


1 


0 


0 


0 


GED 


5% 


2% 








9% 








Brigance 


3 


1 


0 


1 


2 


0 


0 


0 


0 


Assessments 


5% 


2% 




5% 


11% 










SLEP 


1 


0 


1 


0 


1 


0 


0 


0 


0 




2% 




5% 




5% 










TABE-WF 


0 


0 


0 


0 


0 


1 


0 


0 


0 














9% 








API 


0 


1 


1 


0 


0 


1 


0 


0 


0 






2% 


5% 






9% 









Refer to the Key in Appendix C for full names of tests and publisher information 
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Appendix F: 

Other Instruments Used for Other Purposes 



Other Instruments Used for Other Purposes 

Botel Word Opposite Test (Follett) 

TLC Literacy Survey 
Steck- Vaughn GED 

LITSTART (Michigan Literacy Council) 

TABE Work-Related Foundation Skills 

TABE5&6 

Bader 

AMES (Steck-Vaughn) 

Woodcock Reading Mastery Test (AGS - American Guidance Services) 

Ekwall 

MELT 

Side by Side 

Wilson Reading Program (Barbara Wilson) 

Differential Aptitude Test 
BESTEL 

Steck-Vaughn GED 2001 
Slosson 

Stanford Diagnostic Reading Test 

Career Orientation and Evaluation Survey (Edits, San Diego, CA) 

RAT 

Barsch Learning Styles Inventory 
Cooper Screening for Information Processing 
Botel Complete Battery - Inventory (Follett) 

Botel 

Contemporary’s GED Interactive 
TOEFL Practice Test (ETS) 

Laubach Placement/Diagnostic 

Barsch Learning Style Reference Form (Ray Barsch) 

Slosson Intelligence Test (Slosson Publishing Company) 

Glencoe Publishers GED Skills Series Pretest 
Career Ability Placement Survey (Edits, San Diego, CA) 

Peabody 

Contemporary’s Pre-GED Interactive 

Challenger Placement Diagnostic 

Career Interest Survey (Edits, San Diego, CA) 

BETA IQ 
WAIS 
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